The challenges in spectral image analysis: an introduction, and review of ANN approaches

Utilization of remote sensing multiand hyperspectral imagery has been rapidly increasing in numerous areas of economic and scienti c signi cance. Hyperspectral sensors, in particular, provide the detailed information that is known from laboratory measurements to characterize and identify minerals, soils, rocks, plants, water bodies, and other surface materials. This opens up tremendous possibilities for resource exploration and management, environmental monitoring, natural hazard prediction, and more. However, exploitation of the wealth of information in spectral images has yet to match up to the sensors' capabilities, as conventional methods often prove inadequate. ANNs hold the promise to revolutionize this area by overcoming many of the mathematical obstacles that traditional techniques fail at. By providing high speed when implemented in parallel hardware, (near-)real time processing of extremely high data volumes, typical in remote sensing spectral imaging, will also be possible. 1. Challenges in remote spectral image analyses Airborne and satellite-borne spectral imaging has become one of the most advanced tools for collecting vital information about the surface covers of Earth and other planets. The utilization of these data includes areas such as mineral exploration, land use, forestry, natural hazard assessments, water resources, environmental contamination, ecosystem management, biomass and productivity assessment, and many other activities of economic signi cance, as well as prime scienti c pursuits such as looking for possible sources of past or present life on other planets. The number of applications has dramatically increased in the past ten years with the advent of imaging spectrometers, which greatly surpass traditional multi-spectral imagers (e.g., Landsat Thematic Mapper) in that they can resolve the detailed spectral features that are known to characterize minerals, soils, rocks, and vegetation, from laboratory measurements. While a multispectral sensor samples the given wavelength window (typically the 0.4 { 2.5 Work funded by NASA, Applied Information Systems Research Program, NAG54001; Paper in Proceedings of the 7th European Symposium on Arti cial Neural Networks, ESANN99, Bruges, Belgium, April 21{23, 1999, pp 93{98. m range in the case of Visible and Near-Infrared surface re ectance imaging) with several broad bandpasses, leaving large gaps between the bands, spectral imagers (hyperspectral sensors) sample the spectral window contiguously with very narrow badpasses. Figures 1 and 2 illustrate the above. Figure 1 (left). The concept of spectral imaging. Figure from Campbell (1996) [1]. Figure 2 (right). The spectral signature of the mineral alunite as seen through the 6 broad channels of Landsat TM, as seen by the moderate spectral resolution sensor MODIS, and as measured in the laboratory. Hyperspectral sensors such as AVIRIS of NASA/JPL [2] provide spectral details comparable to laboratory measurements. Formally, the vector S = (S 1 ; :::; S NB), where S x;y k is the data value in the kth image band (k = 1; :::; NB) at pixel location (x; y), is called a spectrum. It is a characteristic pattern (Figures 1, 2) which provides a clue to the surface material(s) within pixel (x; y). NB denotes the number of image bands. The feature space spanned by VIS-NIR re ectance spectra is [0; U ] < where U > 0 represents an upper limit of the measured scaled re ectivity. Sections of this space can be very densely populated while other parts may be extremely sparse, depending on the materials in the scene and on the spectral resolution of the sensor. Great spectral detail comes at a cost of a very high data volume, and it also poses new mathematical challenges in the classi cation of images with high spectral dimensionality. The speci c problems associated with remote sensing spectral image analyses arise from any combination of the following [3]: The spectral patterns are high dimensional (dozens NB hundreds); The number of data points (image pixels) can be as large as several millions; The pixels are mixed: Several di erent materials contribute to the spetcral signature detected from each pixel; Given the richness of data, the goal is to separate many cover classes; Di erent surface materials may be distinguished by very subtle di erences in their spectral patterns; Very little training data may be available for some classes; and classes may be represented very unevenly. ANNs have been gaining recognition as powerful answers to the above challenges. It should be emphasized that traditional lower dimensionalmulti-spectral images also bene t greatly from ANN algorithms because remote sensing spectral images of any dimensionality share all but the rst problem above. For this discussion, we will omit additional e ects such as atmospheric distortions, illumination geometry and albedo variations in the scene, because these can be addressed through well-established procedures prior to classi cation. 2. A review of ANN approaches and results In the following, emphasis is on works that overcome mathematical obstacles, or improve classi cation quality over conventional algorithms. In particular, the speed bene t of parallel hardware implementations is not discussed. For sake of space, the reader is referred to the bibliographies in the referenced papers for relevant further works, including well-known ANN paradigms. Traditional multi-spectral images (e.g., Landsat TM) have long been shown to gain improved accuracy from ANN classi cations, using BP networks [4-6], or variants of self-organization, vector quantization, and their hybrids [7-8]. ART networks and variants by [9] were successful in distinguishing vegetation species. Much less work has been done with hyperspectral images, although this type of data would clearly be much better exploited with ANNs than with classical methods. The reason for little work in this area is a combination of the novelty of hyperpsectral imaging (< 10 years in comparison to over 25 years of Landsat), and that the high dimensionality of the input data space requires large, complex networks. [10] presented one of the pioneering papers on simulated 201-band spectra, which were reduced to 20, 40 and 60 bands using feature extraction prior to classi cation into 3 classes. Comparison of several classi ers including Maximum Likelihood (ML), BP network and a Parallel Self-organizing Hierarchical Neural Network (PSHNN) favored the ML, with PSHNN next. However, the authors admitted that the ML had an advantage by virtue of gaussian data generation. [11] successfully classi ed a real AVIRIS image of the Neovolcanic Zone in South-Central Iceland into 9 geological classes, reducing rst the 224 AVIRIS bands to 35. As an important advantage over traditional feature extractors such as PCA, they used an ANN (the same network that performed the classi cation itself) for Decision Boundary Feature Extraction (DBFE). The DBFE is claimed to preserve all features that are necessary to achieve the same accuracy as in the original data space, by a given classi er. Self-Organizing Maps have been recognized as useful tools for classi cation of images with high sepctral dimension. For supervised case, the general observation is that an SOM component in the ANN architecture makes network training much easier (than, e.g., training a BP network); that it produces more accurate classi cation results based on a smaller amount of training spectra than would be required for the training of BP [10]. Using a hybrid SOM-BP architecture, [12] mapped previously undetected soil variants on Mars from 90-band images, [13] improved asteroid compositional taxonomy from 65-band spectra, with no prior feature extraction. Full spectral resolution AVIRIS images were classi ed into large number of output classes by a similar approach [3]. For discoveries in data spaces, SOMs have also been successfully used for the detection of surface compositional classes that were missed by PCA or other conventional techniques [13{14]. MacDonald et al., [15] compare three unsupervised techniques: the Kohonen SOM, the Scale Invariant Feature Map, and the Generative Topographic Mapping, which is a \principled alternative to the SOM". They arrive at similar preliminary results as [13], on the same 65-D data. Since convergence of the GTM can be proven and it has a well-de ned cost function, this investigation may develop into better understanding of hyperspectral spaces than was gained by the above previous works. The mixed pixel problem is addressed by Pendock [16], using an associative ANN to establish a linear mixture model for the areal contributions of \endmember" materials in each pixel. (The endmembers are the spectra whose weighted sum makes up the spectral signature of each pixel. These are typically not the same as the Principle Components of the spectral image.) The linear unmixing approach is one of the most popular conventional techniques in interpreting spectral images [17]. Automated determination of the endmembers, however, has not been very successful. Pendock's approach [16] brings a new solution. In remote sensing, obtaining an ideal number of reliable training samples can be hindered by the inaccessibility of certain locations, or by the fact that small outcrops of important metarials may contain very few recognizable pixels in the scene [3]. This can render some of the most valued conventional classi ers, notably covariance based ones such as ML, useless because those require at least NB + 1 samples for each class [10], [3]. Identi cation of NC (NB + 1) samples, where NC is the number of surface cover classes, can be prohibitively expensive, or impossible, for large NB and NC. Fardanesh and Ersoy [18] o er an architectural approach to compensate for small training sets. Many believe that hyperspectral images are highly redundant because of band correlations. Others maintain an opposite view. Few investigations exist yet into intrinsic dimensionality (ID) of hyperspectral images. Bruske [19] nds the spectral ID of an AVIRIS image to be between 4 and 7, using OptimallyTopology Preserving Maps. This seems consistent with the number of mixture model endmembers in many works, and can be a step toward understanding spectral image compression. The number of separable meaningful spectral classes is an important related question, the answer to which was seen to be more complex when ANNs were utilized than with conventional methods [13{14]. [20] presents a Growing SOM approach applied to Landsat imagery, which can provide more theoretical insight as well as a better practical handle on cluster determination. Visualisation of data clusters in higher-dimensional spaces as detected by SOM type mappings has been targeted by several works [21{24], however, no application to hyperspectral images has been published yet. Mer enyi engineered a tool speci c to hyperspectral data, utilizing [21{22] and [24], and detection of 30 geologically meaningful clusters in an AVIRIS 194-band image from its SOM is demonstrated at http://www.arizona.edu/ erzsebet/annps.html . 3. Future work, outstanding problems Application of ANNs to spectral, especially to hyperspectral imagery is in its infancy. Further, robust solutions are urgently needed to the above, as well as to some other closely related issues (not discussed here) such as classi cation of multi-source disparate data in conjunction with spectral images [9]; missing bands; variable spectral resolution. Image compression has an even more pressing signi cance for multiand hyperspectral data than for monochrome and RGB images. Transmission of enormous data volumes from satellites with limited downlink capacity, and storage of these data merit serious considerations. Encouraging improvement over the most widely used JPEG compression algorithm is presented by Amerijckx et al., [25] using an SOM. Hopefully, such approaches can be extended to multispectral imagery in a way that takes into account and makes use of the band correlations.